Plug-and-Play Attribute-Aware Text Infilling via A New Attention Mechanism and Two-Level Positional Encoding

Text infilling aims to restore incomplete texts by filling in blanks and has attracted increasing attention recently because of its wide application ancient text restoration, conversation generation, rewriting. However, attribute-aware is yet be explored, existing methods seldom focus on the length each blank or number location blanks. In this study, we propose a plug-and-play Attribute-aware Infilling method using Pre-trained language model (A-TIP) that contains text-infilling component discriminator. Specifically, first designed unified with modified mechanisms intra- inter-blank positional encoding better perceive for blank. We then discriminator guide generation improve attribute relevance without decreasing fluency. Finally, automatic human evaluations three open-source datasets indicate that, compared all baselines, A-TIP achieves state-of-the-art performance. An additional ablation study demonstrated robustness A-TIP.